Fine-tuning Transformer Models using Transfer Learning for Multilingual Threatening Text Identification

Threatening content detection on social media has recently gained attention. There is very limited work regarding threatening in low-resource languages, especially Urdu. Furthermore, previous explored only mono-lingual approaches, and multi-lingual was not studied. This research addressed the task of Multi-lingual Content Detection (MTCD) Urdu English languages by exploiting transfer learning methodology with fine-tuning techniques. To address task, we investigated two methodologies: 1) Joint multi-lingual, 2) Joint-translated method. The former approach employs concept building a universal classifier for different whereas latter applies translation process to transform text into one language then perform classification. We explore Multilingual Representations Indian Languages (MuRIL) Robustly Optimized BERT Pre-Training Approach (RoBERTa) that already demonstrated state-of-the-art capturing contextual semantic characteristics within text. For hyper-parameters, manual search grid strategies are utilized find optimum values. Various experiments performed bi-lingual datasets findings revealed proposed outperformed baselines showed benchmark performance. RoBERTa model achieved highest performance demonstrating 92% accuracy 90% macro f1-score joint approach.